Skip to content

Release: Benchmarking improvements and DuckDB comparison#85

Merged
poyrazK merged 2 commits into
mainfrom
release/benchmarking
May 15, 2026
Merged

Release: Benchmarking improvements and DuckDB comparison#85
poyrazK merged 2 commits into
mainfrom
release/benchmarking

Conversation

@poyrazK
Copy link
Copy Markdown
Owner

@poyrazK poyrazK commented May 15, 2026

Summary

  • Add DuckDB comparison benchmark with TPC-H-inspired queries (Q1 GROUP BY, Q6 filter+aggregate, Q3-like join)
  • Add comprehensive performance findings report (docs/performance/DUCKDB_COMPARISON.md)
  • Add tests for error paths in distributed executor, query executor, lock manager, buffer pool, columnar table, bloom filter, and parser
  • Extend CMake to conditionally build DuckDB comparison benchmark when DuckDB is installed

Key Benchmark Findings

  • cloudSQL wins filter+aggregate (Q6): 2.7x–4.5x faster than DuckDB
  • DuckDB dominates GROUP BY (Q1): 385x–1,196x faster than cloudSQL
  • DuckDB leads on hash joins: 9x–18x faster than cloudSQL

Test plan

  • Build all test targets
  • Run distributed_executor_tests
  • Run query_executor_tests
  • Run performance-benchmarks job in CI

poyrazK added 2 commits May 15, 2026 15:50
cloudSQL vs DuckDB comparison using TPC-H-inspired queries:
- Q1: Aggregation with GROUP BY (lineitem scan)
- Q6: Scan with filter (discount + quantity predicate)
- Q3-like: Simple hash join (orders + lineitem)

Benchmark measures items/sec at 10k and 100k row scales.
- benchmarks/duckdb_comparison_bench.cpp: TPC-H-inspired benchmarks
  (Q1 GROUP BY, Q6 filter+aggregate, Q3-like join) at 10k/100k scales
- CMakeLists.txt: conditional DuckDB linking via find_package
- docs/performance/DUCKDB_COMPARISON.md: detailed findings report

Key results:
- cloudSQL wins filter+aggregate (Q6): 2.7x-4.5x faster
- DuckDB dominates GROUP BY (Q1): 385x-1196x faster
- DuckDB leads on joins: 9x-18x faster
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 15, 2026

Warning

Rate limit exceeded

@poyrazK has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 6 minutes and 34 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 041d308e-5f86-47e3-993e-21083cb1ace1

📥 Commits

Reviewing files that changed from the base of the PR and between 5911004 and 96bfd11.

📒 Files selected for processing (3)
  • CMakeLists.txt
  • benchmarks/duckdb_comparison_bench.cpp
  • docs/performance/DUCKDB_COMPARISON.md
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch release/benchmarking

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Owner Author

@poyrazK poyrazK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's okay to merge

@poyrazK poyrazK merged commit fa275e2 into main May 15, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant